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System  Assurance 


•  We  continue  to  be  concerned  with  assurance  of  our 
critical  DoD  assets: 


•  Critical  information 

•  Critical  technologies 

•  Critical  systems 

*  Observations: 


-  Increasing  numbers  of  network  attacks  (internal  and  external  to  DoD) 

-  Broader  attack  space 

•  Trends  that  exacerbate  our  concerns: 

-  Globalization  of  our  contracts,  expanding  the  number  of  international 
participants  in  our  system  developments 

-  Complex  contracting  arrangements  that  further  decrease 
transparency  below  prime,  and  visibility  into  individual  components 


These  trends  increase  the  opportunity  for  access  to  our  critical 

assets,  and  for  tampering 
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Top  Software  Issues 


he  impact  of  requirements  upon  software  is  not  consistently 
quantified  and  managed  in  development  or  sustainment. 

2.  Fundamental  system  engineering  decisions  are  made  without 
full  participation  of  software  engineering. 

3.  Software  life-cycle  planning  and  management  by  acquirers  and 
suppliers  is  ineffective. 


4.  The  quantity  and  quality  of  software  engineering  expertise  is 
insufficient  to  meet  the  demands  of  government  ana  the  defense 
industry. 


5.  Traditional  software  verification  techniques  are  costly  and 

ineffective  for  dealing  with  the  scale  ana  complexity  of  modern 
systems. _ 

6.  There  is  a  failure  to  assure  correct,  predictable,  safe,  secure 
execution  of  complex  software  in  distributed  environments. 


7. 


Inadequate  attention  is  given  to  total  lifecycle  issues  for 
COTS/NDI  impacts  on  lifecycle  cost  and  risk. 

*NDIA  Top  Software  Issues  Workshop 
August  2006 
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System  Assurance  -  Working  Definition 
Level  of  confidence  that  a  system  functions  as  intended,  is  free  of 
exploitable  vulnerabilities,  and  protects  critical  program  information 


Consequences  of  Fragmented  Systems 

Assurance  Initiatives 


•  Lack  of  Coherent  Direction  for  PMs,  and  others  acquiring 
systems 

-  Numerous,  uncoordinated  initiatives 

-  Multiple  constraints  for  PMs,  sometimes  conflicting 

-  Loss  of  time  and  money  and  lack  of  focus  on  applying  the  most 
appropriate  engineering  for  systems  assurance  for  each  system 

•  Synergy  of  Policy  -  Multiple  ownership 

-  Failure  to  capitalize  on  common  methods,  instruction  among 
initiatives 

•  DoD  Risk  Exposure 

-  Lack  of  total  life  cycle  view 

-  Lack  of  a  focal  point  to  endorse  system  assurance,  resolve 
issues,  advocate  PM  attention 

-  Lack  of  system-of-systems,  architecture  perspective  on  system 
assurance 

-  Potential  for  gaps  in  systems  assurance  protection 


Path  Forward 


•  Create  a  ‘framework’  to  integrate  multiple  security  disciplines 
and  policies 

-  Leverage  5200.39:  expand  CPI  definition  to  include  system 
assurance  and  total  life  cycle 

•  Use  the  Program  Protection  Plan  (PPP)  to  identify  CPI  and 
address  assurance  for  the  program 

-  Link  plans  (e.g.,  Anti-Tamper,  Software  Protection,  System 
Engineering,  Assurance  Case) 

•  Modify  Acquisition  and  System  Engineering  guidance  to 
integrate  system  assurance  across  the  lifecycle 

-  Milestone  Decision  Authority  visibility 

-  Guidebook  on  Engineering  for  Assurance  for  program 
managers/engineers 


Raise  the  bar: 

Awareness 

-  Knowledge  of  the  supply  chain 

-  Who  has  access  to  our  critical  assets 

Protection 

-  Protect  critical  assets  through  security  practices  6 

-  Engineer  our  systems  for  assurance 

Policy  Roadmap 
for  System  Assurance 


Current  Systems  Security  Policies 
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Policy  Ownership 

DoD  -  CIO/DSS 
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Proposed  Framework  for  Security  Policies 
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Critical  Pmgram  Information 


New  Definition  -  Draft  DoDI  5200.39: 

•  E3.6.  Critical  Program  Information  (CPI).  Elements  or  components 
of  an  RDA  program  that  if  compromised,  could  cause  significant 
degradation  in  mission  effectiveness,  shorten  the  expected  combat- 
effective  life  of  the  system,  reduce  technological  overmatch, 
significantly  alter  program  direction,  or  enable  an  adversary  to 
counter,  copy,  or  reverse  engineer  the  technology  or  capability. 

•  E3.6.1.  Technologies  become  eligible  for  CPI  selection  when  a 
DoD  Agency  or  military  component  invests  resources  to 
demonstrate  an  application  for  the  technology  in  an  operational 
setting,  or  in  support  of  a  transition  agreement  with  a  Program 
Manager. 

•  E3.6.2.  Includes  information  about  applications,  capabilities, 
processes,  and  end-items. 

•  E3.6.3.  Includes  elements  or  components  critical  to  a  military 
system  or  network  mission  effectiveness. 
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Notional  Assurance  Implementation 


Identify  CPI  in  PPP 

Identify  threats 

Develop  Plans  (AT,  SEP,  TES) 


Approved  SEP  with 
details  on  Assurance 
Milestone  Decision  approves 
plans,  sets  SDD  criteria 
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Sustainment  security  plans  in  place 
Maintenance  providers  meet  security 
practice 

Upgraded  HW/SW  configuration 
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Total  Lifecycle  Approach  to  Assured  Systems 
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Guidebook  on  Engineering 
for  System  Assurance 


SA  Guidebook  intent 


Intent: 

-  Provide  practical  guidance  on  augmenting  systems  engineering 
practice  for  system  assurance 

-  Synthesize  existing  knowledge  from  organizations,  standards 
and  best  practices 

-  Recap  concepts  from  standards 

Implementation: 

-  Iterative  releases  with  updates  as  new  knowledge  is  gained  and 
applied 

-  Multiple  Views  for  information  dissemination 

•  Technical  Project  Manager 

•  System  Engineer 

•  Subject  Matter  Expert  Detail 
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SA  Guidebook  - 
Engineering-in-Depth 


•  Augments  SE  from  documentation  through  engineering  processes 
and  technical  reviews 

-  Introduced  as  early  as  possible  -  Where  there  is  the  greatest  impact 

-  Continue  through  the  life  cycle 

•  Consistent  with  international  standard  and  current  best  practices 

-  E.g.,  Guidebook  approach,  presentation  of  process  /  procedure  consistent 
with  ISO/IEC  15288  standard  for  System  Engineering 

-  Integrates  consideration  and  leverages  numerous  existing  program 
protection  or  security  disciplines  (e.g.,  IA,  AT,  SwA,  SPI,  PPP) 

-  Existing  information  security  /  assurance  material  is  summarized,  and 
leveraged  by  reference,  not  repeated 

•  Test  &  Evaluation;  Center  for  Assured  Software  (CAS) 

•  Enhanced  vulnerability  detection  techniques 

•  SwA  Body  of  Knowledge 

•  Intent  is  to  yield  assured  program  /  system  with  demonstrable 
evidence  of  assurance 
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Standards!  Instructions 
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Future:  Link  to  Acquisition  Guidance,  Evolve/Implement  into  training,  education 
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Related  Standards,  Efforts,  and  Working  Groups. 


Contributors 


•  NDIA 

•  INCOSE 

•  MITRE 

•  IDA 

•  SEI 

•  OSD,  Joint  Staff,  Services 

•  Contractor  community 

•  Academe 
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Milestones  &  Plan 


•  Complete  the  Guidebook 

-  Increment  versions  through  Summer,  2007 

-  Focus:  “Get  the  content  right”... worry  format  and  organization 
later 

•  Stakeholder  Review 

-  From  the  larger  community,  different  perspectives 

•  Pilots 

-  Systems  Assurance  innovators  and  areas  where  comprehensive 
expertise  in  one  or  more  relevant  domains  exists 

-  Starting  Summer,  2007 

•  Write  SE,  PM,  ISSE/IA  Views 

-  Focus:  Derived  from  the  Guidebook,  “get  the  right  content”  (by 
audience) 

•  Release  version  0.9  by  30  September 


Contact  us  to  participate  in  stakeholder  review 
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Community  Site 
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Systems  Assurance  Committee 


Mission 

Asaura  af**ct>v#  functionary  </  our  ccrrrwid,  ccr t'xJ,  cammcricat  or*  aid  raiatad 
«aapci n  tytUrn  «iti  hkj'i  cGrfidenc*  th *t  th«  tytfa'ns  an  ret  unata&«  to  intnaioc  arc 
cannot  bo  campromsad  by: 

•  Estatlbhug  mamUrtho  from  across  *1  oonmun  flies  cl  ntsns; 

o  Da'a'aa  industry  system  it^  ato's  a  >J  ttbeortrseters 
•  Ccmnarcial  Indust ->  (component  smppH*'*) 
o  Non  dafarsa  ndstry  system  angtiaars/irtagrator* 

•  Captir mg  arrat  industry  practices 

•  rUiating  a  SfStam  As  nuance  Wita  Paper 

o  Daf intern  c/  System  laturann  Pro&an 
o  Systems  argnaa'irc  community  gcab 

•  Dev#  oping  a  S*«arr  Assonance  Hardteok 

o  Practical  guideica 

o  Targeted  *or  aegubflion  prp*ess<cnab  and  Program  Maneg*  t 

•  Devacping  a  plan  fer  la*«<  agng  ralavart  standards  and  idaitlfyug  gaps 


Committee  Co-Chairs: 

Hr.  Pad  Cxll 
Compete'  Sciences  Corp. 

(540)644  6224 

pci  a  1^; sc.  com 

Hi  Kiitlan  Balia>i 
CUSDlATS.i  CS/Sr 
(703)605  2300 

tiataii  paic%»<i 

Hr.  Mflcft  <cmaroHT 

qasd :  sni> 

(703)602  0000  Ext.  146 

htibJbLlJcL  nidi  .»*!  fr.fr J  .mil 


•  Presidents  Ccmer 

NQIA  Resources 

•  NeO:n*  D*m*t 

•  likii  "laom 
•Lrx  CM 

•  Suiaca  Resorts 

6  Proosecirgs 

Vol  Our  Bsn-muru 


Detection 

Technologies 


/ 


Results  and 
com  prance 
mroo^i 
business 
nteltgence 

Click  here  to 
knd  out  more. 


- 


http://tin  yurl.  com/222h  vg 


Committee  L'nks 

p^i  Mima 

Stje-m-tj  GLidaaook 

r.j  Cdx.i.k  AjUC hdii. 
a J  AiUtk  Mq  I M  r.i* 

S-iduUXk 

Sviia-Yj  Writa  Papa- Frolaci 


19 


System  Assurance: 

What  does  success  look  tike? 


The  requirement  for  assurance  is  allocated 
among  the  right  systems  and  their  critical 
components 

DoD  understands  its  supply  chain  risks 

DoD  systems  are  designed  and  sustained 
at  a  known  level  of  assurance 

Commercial  sector  shares  ownership  and 
builds  assured  products 

Technology  investment  transforms  the 
ability  to  detect  and  mitigate  system 
vulnerabilities 


Backups 
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Fragmented  Systems  Security  Policies 


Each  policy: 

•  Affects  different  parts  of  the  life 
cycle 

-  R&D,  acquisition,  foreign  ownership 

•  Applies  to  a  different  subset  of  DoD 
systems 

-  NSS,  IT,  MDA,  ACAT  1 C,  etc. 

•  Assures  different  ‘type’  of 
components 

-  information,  leading  technology, 
functionality 

•  Mandates  a  different  set  of  defense 
tactics 

-  intelligence,  engineering,  documented 
plan,  certification  &  accreditation 


CC  -  Common  Criteria 

DIACAP  -  DoD  Certification  & 
Accreditation 

FIPS  -  Federal  Information  Processing 
Standards 

ITAR  -  International  Traffic  in  Arms 
Regulation 

IA  -  Information  Assurance 

ISP  -  Information  Security  Program 

NIAP  -  National  Information  Assurance 
Partnership 

NISP  -  National  Industrial  Security 
Program 

OPSEC  -  Operational  Security 

5200.39  -  DODD  5200.39  Security, 
Intelligence,  and  Counterintelligence 
Support  to  Acquisition  Program 
Protection 

SA  -  System  Assurance 

SPI  -  Software  Protection  Initiative 

TF  -  Trusted  Foundry 


Current  approach  does  not  have  systems-of-systems  perspective 
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